Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

We study the problem of estimating an unknown function from noisy data using shallow ReLU neural networks. The estimators we minimize sum squared data-fitting errors plus a regularization term proportional to Euclidean norm network weights. This minimization corresponds common approach training with weight decay. quantify performance (mean-squared error) these when data-generating belongs second-order Radon-domain bounded variation space. space functions was recently proposed as natural associated derive minimax lower bound for estimation this and show that are optimal up logarithmic factors. rate is immune curse dimensionality. explicit gap between networks linear methods (which include kernel methods) by deriving problem, showing necessarily suffer dimensionality in As result, paper sheds light on phenomenon seem break